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1.  INTRODUCTION 


The  bootstrap,  so  named  by  Efron  *  to  convey  its  self-help  attributes,  is  a  conceptually 
simple  technique.  It  is  one  of  a  number  of  procedures  known  as  resampling  plans  whose  goal  is 
to  extract  information  from  a  set  of  data  through  repeated  iaspection.  Although  the  theoretical 
foundation  is  incomplete  and  many  of  the  available  results  are  empirical,  this  Ls  an  area  of 
active  research  that  will  grow  in  importance  as  computation  becomes  faster  and  cheaper. 


The  bootstrap  attempts  to  address  an  important  statistical  problem;  having  computed  an 
estimate  of  some  parameter,  say  a  mean  or  a  probability  or  a  correlation,  what  accuracy  can  be 
attached  to  the  estimate?  Accuracy  refers  to  the  "±  something"  that  often  accompanies 
statistical  estimates  and  is  commonly  conveyed  through  such  devices  as  variance,  standard 
error,  confidence  interval,  etc.. 

Before  detailing  the  mechanics  of  the  bootstrap  procedure,  it  is  instructive  to  study  the 
follOMtig  simple  example. 

Consider  a  data  set  consisting  of  a  random  sample  of  size  n  from  an  unknown  probability 
distribution  F  on  the  real  line,  X X2  >  •  ^  ‘  Having  observed 

_  1  ” 

Xi  =Xj,  A'2=JC2  >  •  ■  ■  >  sample  meanX  ■=  —  X  is  computed  for  use  as 

n 

an  estimate  of  the  distribution  mean  fi. 


A  crucial  fact  is  that  the  sample  values  Xj,  X2  .....  X„  contain  more  information 
than  the  estimate  X.  They  also  provide  an  estimate  for  the  accuracy  of  X,  namely 


/i(«— 1) 


(1) 


the  staixlard  error  of  X.  Unfortunately,  equation  (1)  has  no  obvious  extension  to  estimators 
other  than  X.  The  bootstrap  is  one  methcxl  of  making  this  extensioiL 


Let  F  be  the  empirical  distribution  of  the  data  with  probability  mass  —  assigned  to  each 

n 

X  j  and  let  ,^2  ,  .  .  .  ,  X„  be  a  ratxiom  sample  from  F.  In  other  words,  each  X;  is 

drawn  independently  with  replacement  from  the  set  {xj,  X2.  .  .  .  tX„].  The  bootstrap 
estimate  of  standard  error  for  an  estimator  OiXi^  X2  X„)  is 

1 

&B  =  {Var  [tai*,  X2\  .  •  •  ,X„*)]}^.  (2) 


^  B.  EJron,  'Bootstrap  methods:  another  look  at  the  Jackknife, "Ann.  Statist.,  7(1979),  pp  1-26. 
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Comparisfjn  of  (2)  with  (1 )  shows  that 


n 

n-1 


for  6  =  X  ; 


i.e.,  o’^  is  a  biased  estimator. 


(3) 


Consider  the  random  sample  X X2  >  ■  ■  •  •  ~  ^  ^  estimator 

A  A 

OiXiy  X2  ,  .  •  •  ,  X„).  The  standard  error  of  6  depends  on  the  distribution  F,  i.e., 

1 

=  [  VoTf  9iX X 2  ,  .  .  .  ,  X„)  ]^ .  It  also  depends  on  the  sample  size  n  and  the 

A 

functional  form  of  9,  both  of  which  are  known  What  is  not  known  is  F,  but  it  can  be 
estimated  by  F.  The  bootstrap  estimate  of  (t(F^  is  then  or{F),  as  expressed  in 

equation  (2). 

In  most  cases  of  interest  fr{F)  is  impossible  to  express  in  closed  form.  Fortunately, 
may  be  approxirriated  by  Monte  Carlo  methods  as  follows: 

A 

(i)  Construct  the  empirical  disuibution  F  from  the  sampled  values  {Xj,  X2  ,  .  .  .  ,  X „}; 

%  #  ^  A 

(ii)  Draw  a  bootstrap  sample  Xi  ,  X2  ,  ,  X„  by  sampling  with  replacement  from  F 

and  calculate  ,  X2  ,  ....  X„  )\ 

(iii)  Repeal  (ii)  B  limes,  obtaining  independent  bootstrap  replications  ^ 

and  finally. 


(iv)  Compute 


where  9 


Est&g 


1 


B 

z 


1  ^ 

—  As  B  — *0®,  (4)  approaches  (2). 


(4) 


The  bootstrap  procedure  is  far  more  general  than  has  been  irxlicated  As  a  matter  of  fact, 
the  standard  error,  which  has  been  the  focus  of  attention,  need  not  play  a  pivotal  role.  To 
illustrate  this,  consider  a  general  one-sample  problem  in  which  ZiX ,F)  is  a  random  variable 
and  the  vector  I,  X2  >  >  X„)  represents  a  random  sample  from  the  distribution 

F.  Based  on  an  observed  X^X,  some  aspect  of  the  distribution  of  Z,  e.g., 
EfZ,  Prf[Z<.z],  Z  50  ,  is  to  be  estimated.  The  bootstrap  procedure  (i)-(iv)  remains 
essentially  unchanged;  at  step  (ii)  Z  “ZW  ^F)  is  calculated  as  9  and  at  step  (iv)  the  aspect 
of  the  distribution  of  Z  which  is  of  interest  is  calculated,  rather  than  an  estimate  of  standard 


10 


error.  An  application  of  the  bootstrap  procedure  to  confider>cc  intervaJ  coastruction  wll  be  the 
sub.ieci  of  Section  3  of  this  report. 


2.  ESTIMATION  OF  VULNERABLE  AREA  AND  PROBABILITY  OF  KILL 


Consider  the  conceptual  item  of  military  hardware  illustrated  in  Figure  1.  Without  loss  of 
getKrality,  the  component  edges  are  assumed  to  be  aligned  with  the  superimposed  30x30 
rectangular  grid.  The  color  in  the  /th  cell  represents  the  probability  Pk\h,  target  will 

be  killed  should  a  prescribed  round  of  ammunition  Qred  from  a  fixed  range  and  orientation 
impact  within  that  cell. 


The  vulnerable  area  A  ^  for  the  target  is  calculated  by 

V  '^^Pk\h,^  i  “  ^  S  ^A:  |A, 

/■  i 


(5) 


where  /  is  indexed  over  the  cells  of  the  grid.  In  this  example,  the  cell  areas  are  identical,  Le., 
^  A.  The  overall  probability  of  kill  is 


Pk 


(6) 


where  A  p  \s  the  total  presented  area  of  the  target  For  the  conceptual  target  shown  in 
Figure  1,  the  exact  values  of  vulnerable  area  and  probability  of  kill  are  Ay*=  188.1  and 
P,  -  0.21. 


In  practice,  the  values  are  unknown,  and  estimates  Pk\h,  ^  required  to  obtain  the 
approximations 

Ay  A  '^Pk\h,  > 

/ 


The  method  by  which  the  values  Pk\h.  ^  produced  will  not  be  considered  here.  Determining 
valid  Pk\h,  estimates  is  a  persistent  problem  because  of  the  difficulty  in  modeling  the 
underlying  damage  mechanisms  and  constitutes  an  additional  source  of  uncertainty.  The 
procedure  used  here  assumes  the  individual  Py^  ^ ’s  are  accurately  represented  aixi  does  not 
attempt  to  assign  a  component  of  variation  due  to  this  factor. 
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3.  THE  PERCENTILE  METHOD  FOR  CONFIDENCE  INTERVAL  CONSTRUCTION 


The  booLStrap  procedure  was  applied  to  consiruci  an  approximate  confidence  interval  for 
the  parameter  P/^.  A  percentile  method  ^  was  used  which  allows  an  approximate  confidence 
interval  to  be  assigrxid  to  any  real-valued  parameter  9  =  0{F)  based  on  the  bootstrap 
distribution  of  0  —  0{F^. 

Let 

ht)--Pr*{9  ^t]  (9) 


A* 

be  the  probability  distribution  of  9  .  For  0  ^  a  ^  0.5  ,  define 

0„p(at)  •=  /“Hi— a).  (10) 

An  approximate  1  — 2af  central  confidence  interval  for  9  may  then  be  chosen  as 

],  (11) 

which  is  the  central  1  — 2a  portion  of  the  distribution  of  bootstrapped  9  .  Under  Monte  Carlo 
sampling, 


/(()=== 


N{9^ 

B 


(12) 


where  the  operator  N{  }  denotes  the  cardinality  of  the  set  This  approximation  was  used  to 
evaluate  the  confidence  interval  (11). 


i.  APPLICATIONS 


(4. 1 )  Surrogate  Target 

The  percentile  method  was  applied  to  construct  a  confidence  interval  for  the  parameter 
Pf^  corresponding  to  the  target  in  Figure  1.  A  rarxlom  location  within  each  cell  of  the  30x30 
grid  was  selected  and  the  corresponding  Pk\h,  determined.  This  is  consistent  with  vulnerability 
methodology  but  represents  a  departure  from  step  (ii)  of  the  usual  bootsuap  procedure  in  that 
sampling  within  each  grid  cell  produces  a  stratified  sample  with  one  observation  per  strata  arri 
ret  a  simple  random  sample.  This  example  was  contrived  to  provide  an  instance  in  which  the 
true  parameter  value  {P/^  —  0.21)  was  known,  and  so  the  adopted  strategy  was  to  proceed  as  if 
was  unknown  and  observe  how  robust  the  bootstrap  procedure  was  to  departure  from  the 
usual  sampling  scheme. 


2 

B.  Ffivn.'The  Jackknife,  the  Bootstrap  and  Other  Resampling  Plans, "SIAM Monograph  38(1982). 
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After  determining  a  rarxiom  point  of  impact  and  its  corresponding  Pi(\hi  /  ^ 

the  nine  hundred  Pk\h,  fornaecl  the  population  which  was  then  sampled  with  replacement  to 
determine  a  bootstrap  estimate  Pf^  of  the  target  parameter  Pf^ .  Estimate  P|^  was  based  on  a 
sample  size  equal  to  the  number  of  cells  in  the  overlaid  grid.  The  procedure  of  sampling  with 
replacement  was  repeated  until  one  thousarxi  bootstrap  estimates  ,  were  obuiined.  From 
the  one  thousand  ordered  estimates  Pf^  Pk  ’  ^Pk  a  single 

confidence  interval  for  Pf^  was  coastructed  by  the  percentile  method  . 

Since  the  overlaid  grid  and  the  vulnerability  grid  coincided  in  this  example,  repetition  of 
the  procedure  beginning  with  determination  of  random  within-cell  locations  would  merely 
generate  an  identical  population  and  no  additional  information  would  be  gained.  Therefore,  the 
mesh  of  the  overlaid  grid  was  coarserxjd  to  induce  w-ithin-cell  variation.  The  procedure  detailed 
in  the  preceeding  paragraph  was  then  repeated  one  hundred  times  to  determine  one  hundred 
(not  necessarily  distinct)  populations  from  which  to  sample  with  replacement. 

In  general,  for  an  overlaid  grid  of  size  nxn,  one  hurtdred  populations  were  determined. 
From  each  of  the  one  hundred  pop'ulations,  one  thousand  bootstrapped  were  generated 
and  a  corresponding  confidence  interval  computed.  Each  bootstrapped  Pj^  was  based  on  a 
sample  of  size  n^.  A  flow  chart  of  the  computation  is  drawn  in  Figure  2. 

An  example  of  one  hundred  70%  confidence  intervals  constructed  for  the  for  the 

target  in  Figure  1  is  shown  in  Figures.  Theoretically,  a  100(1 — a)%  confidence  interval 
covers  the  true  parameter  value  100(1 — 0:)%  of  the  time.  In  this  example,  the  percentile 
method  of  confiderxie  interval  construction  performs  more  conservatively  than  theory  would 
suggest;  the  true  Pf^  was  covered  91%  of  the  time.  We  suspect  this  is  a  consequence  of  the 
stratified  sampling  plan,  but  this  remains  an  unproven  coryecture. 

The  results  of  a  simulation  study  for  selected  coarser  grids  on  the  target  in  Figure  1  are 
summarized  in  Tables  1  and  2  and  displayed  graphically  in  Figure  4.  The  entries  in  the  body  of 
Table  1  are  the  number  of  confidence  intervals  constructed  by  the  percentile  method  that 
actually  covered  the  true  parameter  value;  e.g.,  for  a  grid  of  size  10x10  the  confidence  interval 
[P^^^(.IO)  ,  P)t(-90)]  covered  the  true  parameter  Py^  ninety-six  of  a  possible  one  hundred 
times.  The  average  width  of  the  one  hundred  confidence  intervals  appears  at  the  corresponding 
location  wnthin  Table  2  and  for  this  example  is  .080.  Notice  again  the  apparent 
conservativeness  of  this  procedure  for  the  finer  grid  sizes;  in  this  instance,  an  estimated  80% 
confidence  interval  behaves  like  a  %%  confidence  interval. 
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Determine  random  location  and 


corresponding  for  each  cell  of  grid 


Figure  2.  Flow  chart  for  the  bootstrap  computation. 
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Figure  3.  One  hundred  bootstrapped  confidence  intervals. 


TABLE  1.  NUMBER  OF  CONFIDENCE  INTERVALS  COVERING  THE  PARAMETER 

SURROGATE  TARGET 

Grid 

Inner  Percentile  Range  (Central  Area) 

Size 

.99 

.95 

.90 

.70 

.30 

.20 

.10 

15x15 

100 

100 

100 

100 

93 

88 

80 

68 

53 

38 

15 

10x10 

100 

100 

99 

96 

91 

80 

72 

57 

46 

31 

18 

6x6 

100 

100 

99 

91 

84 

72 

59 

46 

34 

26 

8 

5x5 

99 

95 

93 

86 

79 

71 

58 

46 

32 

24 

15 

3x3 

93 

91 

88 

75 

66 

61 

49 

38 

29 

19 

6 

2x2 

76 

76 

69 

60 

58 

48 

43 

28 

9 

5 

1 
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TABLE  2.  MEAN  WIDTH  OF  CONFIDENCE  INTERVALS 
SURROGATE  TARGET 


Grid 

Size 

.99 

.95 

.90 

Inner  Percentile  Range  (Central  Area) 
.80  .70  .60  .50  .40 

.30 

.20 

.10 

15x15 

.109 

.082 

.069 

.054 

.044 

.036 

.028 

.022 

.016 

.011 

.005 

10x10 

.162 

.122 

.103 

.080 

.065 

.053 

.042 

.033 

.024 

.016 

.008 

6x6 

.265 

.202 

.169 

.132 

.107 

.087 

.070 

.054 

.040 

.026 

.013 

5x5 

.315 

.241 

.204 

.159 

.128 

.104 

.084 

.065 

.048 

.031 

.016 

3x3 

.467 

.367 

.315 

.250 

.197 

.166 

.139 

.106 

.073 

.042 

.020 

2x2 

.514 

.453 

.405 

.289 

.278 

.246 

.226 

.134 

.049 

.027 

.007 

Figure  4  requires  some  explanation;  displayed  there  are  the  combined  data  listed  in  Tables 
1  and  2.  Plotted  along  the  x-axis  is  the  mean  width  of  the  confidence  interval  appearing  in  the 
body  of  Table  2;  plotted  along  the  y-axis  are  the  corresponding  empirical  confidence  levels 
obtained  from  the  entries  of  Table  1.  Each  segmented  line  in  Figure  4  corresponds  to  a 
different  grid  size.  The  lines  are  ordered,  the  bottommost  corresponding  to  the  coarsest 
grid  (2x2)  up  through  the  topmost,  which  corresponds  to  the  finest  grid  (15x15). 


Finally,  along  each  segmented  line,  the  different  symbols  correspond  to  the  inner 
percentile  ranges  that  appear  in  both  Tables  1  and  2.  The  key  for  the  percentile  range  symbols 
(which  will  be  used  hereafter  in  the  presentation  of  data)  appear  below. 
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(4.2)  Armored  Personnel  Carrier 


The  largci  descripuon  shown  in  Figure  5  was  used  for  a  .sccorxi  applieaiion  of  the 
booisuap  procedure.  This  is  a  represeniaiion  of  an  armored  personnel  earner  with  a 
superimposed  48x120  vuIrKrability  grid.^  The  finest  mxn  overlaid  grid  for  the  sa'iij'ntion  study 
was  24x60;  coarser  grids  of  dimeasion  16x40,  12x30,  8x20,  6x15,  4x10,  arxJ  2x5  \\tre  studied. 
As  in  example  4.1,  one  thousand  boousirappcd  estimates  of  were  used  to  establish  each  of 
the  one  hundred  O’s.  Each  of  the  one  thousand  estimates  Pj^  was  based  on  a  sample  of 
size  ma  Tables  3  -  4  and  Figure  6  arc  analogous  to  Tables  1  -  2  and  Figure  4  of  example  4.1. 
The  method  of  constructing  approximate  confidence  intervaJs  continues  to  appear  conservative. 
Notice,  in  mis  example,  the  finest  grid  was  not  required  to  produce  highly  significant  confidence 
intervals  with  small  expected  widths;  e.g.,  for  a  16x40  grid,  the  .90  inner  percent De  range 
covered  the  parameter  every  time  with  a  mean  width  of  .053. 


TABLE  3.  NUMBER  OF  CONFIDENCE  INTERVALS  COVERING  THE  PARAMETER 

ARMORED  PERSONNEL  CARRIER 


Grid 

Size 

.99 

.95 

.90 

Inner  Percentile  Range  (Central  Area) 
.80  .70  .60  .50  .40 

.30 

.20 

.10 

24x60 

100 

100 

100 

100 

100 

98 

87 

74 

61 

49 

28 

16x40 

100 

100 

100 

99 

98 

94 

87 

80 

65 

55 

35 

12x30 

100 

100 

100 

100 

95 

88 

79 

68 

51 

37 

22 

8x20 

100 

99 

99 

95 

88 

80 

68 

56 

40 

30 

15 

6x15 

100 

100 

97 

90 

83 

75 

66 

52 

45 

30 

20 

4x10 

100 

96 

94 

85 

78 

70 

62 

50 

39 

24 

17 

2x5 

98 

94 

91 

80 

66 

59 

50 

37 

31 

19 

10 

TABLE  4.  MEAN  WIDTH  OF  CONFIDENCE  INTERVALS. 
ARMORED  PERSONNEL  CARRIER 


Grid 

Size 

.99 

.95 

.90 

Inner  Percentile  Range  (Central  Area) 
.80  .70  .60  .50  .40 

.30 

.20 

.10 

24x60 

.056 

.042 

.035 

.027 

.022 

.018 

.014 

.011 

.008 

.005 

.003 

16x40 

.084 

.063 

.053 

.041 

.033 

.027 

.022 

.017 

.012 

.008 

.004 

12x30 

.112 

.084 

.071 

.055 

.044 

.036 

.029 

.023 

.016 

.011 

.005 

8x20 

.167 

.127 

.106 

.083 

.067 

.054 

.043 

.034 

.025 

.016 

.008 

6x15 

.221 

.168 

.141 

.109 

.089 

.072 

.058 

.045 

.033 

.022 

.011 

4x10 

.330 

.250 

.210 

.163 

.132 

.108 

.086 

.067 

.049 

.033 

.016 

2x5 

.615 

.477 

.403 

.316 

.258 

.208 

.167  • 

.131 

.097 

.062 

.029 

3 


This  schematic  was  provided  by  L.D.  Lose.  VLD,  BRL 
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21! 


(4.3)  Tank 

The  most  detailed  target  description  considered  was  selected  for  a  third  applicatioa  This 
target  appears  in  Figure  7  with  a  superimposed  24x64  vulnerability  grid.^  The  procedure  for 
constructing  approximate  confidence  intervals  remains  unchanged;  however,  the  way  in  which 
the  compulation  (weighted  and  unweighted)  of  target  will  be  treated 
as  two  cases  and  discussed  separately. 


4 


This  schematic  was  provided  by  H.W .  Ege  and  LD.  Lose,  VLD,  BRL. 
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Figure  7.  Tank  target  description. 


(4.3.1)  Unweighted  Procedure 


The  unweighted  procedure  was  used  in  examples  4.1  and  4.2.  A  24x64  vulnerability  grid 
has  been  superimposed  on  the  target  and  the  Pk\h,  “cupy  corresponding  cell  locations. 
One  thousand  bootstrapped  estimates  of  were  used  to  determine  each  of  or>e  hundred  O’s. 
Grid  cells  that  were  dctermirxid  to  be  empty  (the  random  within-cell  locauon  fell  off  the  target) 

A 

were  igixired  when  calculating  the  individual  P^.,  so  the  sample  size  corresponding  to  an 

A 

estimate  is  rx)  longer  mn  but  mn  less  the  number  of  empty  cells.  This  value  is  of  course 
dependent  upon  the  grid  size  as  well  as  the  random  within-cell  locations. 

The  data  summary  for  this  part  of  the  study  is  contained  in  Tables  5-6  and  Figure  8. 


TABLE  5.  NUMBER  OF  CONFIDENCE  INTERVALS  COVERING  THE  PARAMETER 

UNWEIGHTED  PROCEDURE 


Grid 

Size 

.99 

.95 

.90 

Inner  Percentile  Range  (Central  Area) 
.80  .70  .60  .50  .40 

.30 

.20 

.10 

12x32 

100 

100 

100 

100 

99 

94 

90 

79 

64 

55 

34 

6x16 

100 

99 

98 

91 

84 

76 

68 

57 

43 

33 

14 

3x8 

98 

98 

98 

93 

86 

76 

65 

60 

47 

29 

16 

TABLE  6.  MEAN  WIDTH  OF  CONFIDENCE  INTERVALS. 
UNWEIGHTED  PROCEDURE 


Grid 

Size 

.99 

.95 

.90 

Inner  Percentile  Range  (Central  Area) 
.80  .70  .60  .50  .40 

.30 

.20 

.10 

12x32 

.111 

.083 

.054 

.044 

.036 

.029 

.022 

.016 

.011 

.005 

6x16 

.218 

.165 

.138 

.108 

.088 

.071 

.057 

.044 

.032 

.021 

.011 

3x8 

.431 

.332 

.278 

.217 

.177 

.144 

.115 

.091 

.066 

.044 

.022 
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(4.3.2)  Weighted  Procedure 

Up  to  now,  the  tacit  assumption  has  been  that  although  Pk\hi  change  drastically 
from  cell-io-cell,  the  probability  of  impact  within  all  cells  is  the  same.  Tn  other  words,  the 
rounds  fall  on  the  target  according  to  a  bivariate  uniform  distribution.  The  unweighted 
procedure  for  esiimatirig  Pf^  is  modified  for  the  target  in  Figure  7  when  an  aim  point  on  the 
target  is  designated  and  the  Pf^  ’s  are  weighted  by  the  probability  of  their  occurrence. 

The  weight  associated  with  an  individual  Pk\h,  obtained  by  determining  the  probability 
Pf^  ^  that  a  round  will  impact  in  the  /  th  cell  if  the  rounds  fall  according  to  a  bivariate  normal 
distribution  centered  at  the  aim  point  and  having  variance  cr^.  The  computational  formula  (5) 
becomes 

^  V Pk\h,^h, 

with 

Ap-=A^Ph^  (14) 
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Confidence  Level 


The  data  summary  for  the  target  in  the  weighted  case  is  contained  in  Tables  7-8  and 
Figure  9  for  cr“=l,  Tables  9-10  and  Figure  10  for  O'— 3,  and  Tables  11  -  12  and  Figure  11  for 
cr-9. 


TABLE  7.  NUMBER  OF  CONFIDENCE  INTERVALS  COVERING  THE  PARAMETER 

WEIGHTED  PROCEDURE,  o-=l 


Grid 

Size 

.99 

.95 

.90 

Inner  Percentile  Range  (Central  Area) 
.80  .70  .60  .50  .40 

.30 

.20 

.10 

12x32 

93 

83 

73 

63 

52 

48 

38 

27 

22 

14 

7 

6x16 

68 

49 

40 

32 

25 

16 

13 

9 

5 

4 

3 

3x8 

63 

58 

55 

48 

44 

35 

31 

22 

17 

11 

4 

TABLE  8.  MEAN  WIDTH  OF  CONFIDENCE  INTERVALS 
WEIGHTED  PROCEDURE,  o'- 1 


Grid 

Size 

.99 

.95 

.90 

Inner  Percentile  Range  (Central  Area) 
.80  .70  .60  .50  .40 

.30 

.20 

.10 

12x32 

.090 

m 

.045 

.036 

.029 

.023 

.018 

.013 

.009 

.004 

6x16 

.088 

.071 

.058 

.046 

.036 

.026 

.017 

.009 

3x8 

.169 

.138 

.112 

.089 

.069 

.051 

.033 

.016 
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TABLE  9.  NUMBER  OF  CONFIDENCE  INTERVALS  COVERING  THE  PARAMETER 

WEIGHTED  PROCEDURE,  a =3 


Grid 

Size 

.99 

.95 

.90 

Inner  Percentile  Range  (Central  Area) 
.80  .70  .60  .50  .40 

.30 

.20 

.10 

12x32 

100 

93 

85 

71 

62 

50 

44 

34 

22 

16 

7 

6x16 

99 

97 

92 

83 

70 

55 

48 

38 

31 

21 

10 

3x8 

93 

8S 

80 

69 

65 

58 

46 

39 

28 

23 

13 

TABLE  10.  MEAN  WIDTH  OF  CONFIDENCE  INTERVALS 
WEIGHTED  PROCEDURE,  tr-3 


Grid 

Size 

.99 

.95 

.90 

Inner  Percentile  Range  (Central  Area) 
.80  .70  .60  .50  .40 

.30 

.20 

.10 

12x32 

.115 

.086 

.072 

.056 

.045 

.037 

.030 

.023 

.017 

.011 

.005 

6x16 

.230 

.173 

.145 

.113 

.092 

.074 

.060 

.047 

.034 

.023 

.011 

3x8 

_ 1 

.418 

.319 

.269 

.210 

.170 

.138 

.111 

.086 

.063 

.042 

.021 
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TABLE  11.  NUMBER  OF  CONFIDENCE  INTERVALS  COVERING  THE  PARAMETER 

WEIGHTED  PROCEDURE,  o-=9 


Grid 

Size 

.99 

.95 

.90 

Inner  Percentile  Range  (Central  Area) 
.80  .70  .60  .50  .40 

.30 

.20 

.10 

12x32 

100 

100 

100 

93 

88 

79 

67 

55 

40 

27 

14 

6x16 

100 

99 

99 

96 

88 

83 

66 

50 

41 

34 

18 

3x8 

96 

95 

89 

84 

75 

71 

63 

58 

46 

33 

20 

TABLE  12.  MEAN  WIDTH  OF  CONFIDENCE  INTERVALS 
WEIGHTED  PROCEDURE,  o-=9 


Grid 

Size 

.99 

.95 

.90 

Inner  Percentile  Range  (Central  Area) 
.80  .70  .60  .50  .40 

.30 

.20 

.10 

12x32 

IQ 

.083 

.070 

.054 

.044 

.036 

.029 

.022 

.016 

.011 

.005 

6x16 

.164 

.137 

.107 

.087 

.070 

.057 

.044 

.032 

.021 

.011 

3x8 

.306 

.258 

.202 

.163 

.133 

.107 

.083 

.061 

.040 

.020 
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When  ihe  rounds  fall  according  lo  a  bivariate  uniform  distribution  the  bootstrap  procedure 
appears  very  conservative,  generating  intervals  which  cover  the  parameter  with  a  frequeixy 
in  excess  of  the  theoretical  value.  In  the  bivariate  normal  setup  with  values  of  cr^l,  3  the 
bootstrap  procedure  continues  to  perform  well  but  the  confidence  intervals  are  no  longer 
conservative  for  the  overlaid  grids  considered,  as  can  be  seen  in  Tables  7  and  9.  However  for  cr 
=  9,  where  the  normal  distribution  begins  to  behave  more  like  a  uniform  distribution  over  the 
target,  the  conservativeness  reappears  for  finer  grid  sizes. 


5.  SUMMARY  AND  CONCLUSIONS 


The  bootstrap,  a  computer-intensive  procedure  for  data  analysis,  was  applied  to  an 
estimation  problem  to  enable  a  statement  to  be  made  about  the  variability  inherent  in  a 

A 

probabiliiy-of-kill  estimate  Pf^.  The  bootstrap  was  applied  to  a  stratified  sample  rather  than  a 
simple  random  sample  and  its  performance  evaluated  in  this  framework,  first  in  an  abstract 
situation  where  the  parameter  Pf^  was  known  and  then  in  three  situations  where  the  parameter 
Pf^  was  unknown,  but  an  estimate  provided  by  current  vulnerability  analysis  procedures  was 
available.  The  investigation  was  carried  out  for  several  grid  sizes,  or  alternatively,  for  several 
levels  of  detail,  to  study  the  effect  of  grid  size  on  the  estimation  of  Pf^  and  the  reliability  of  the 

A 

confidence  intervals  constructed.  Uncertainty  inherent  in  the  individual  ’s  was  not 

addressed  in  this  study. 

It  was  pointed  out  that  when  the  overlaid  grid  and  vulnerability  grid  coincide,  only  a 
single  bootstrap  confidence  interval  will  be  produced.  In  practice,  this  is  the  situation  that  is 
likely  to  be  of  most  interest,  since  it  exploits  the  highest  resolution  of  detail.  In  this  situation  a 
single  confidence  interval  is  sufficient 

This  investigation  was  exploratory  in  nature,  and  conclusions  are  tentative,  in  the  sense 
that  they  were  drawn  from  empirical  evidence.  Howver,  the  available  evidence  strongly 
suggests  the  use  of  the  bootstrap  procedure  to  provide  an  interval  estimate  to  accompany  the 
conespording  point  estimate  • 
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